WK3 Decision Support Systems for Business Intelligence CH4

4

MODEL COMPONENT

Business Intelligence allows decision makers to have a better understanding of the context

of their choices. It is based upon the collection and examination of information called

"analytics." Analytics are the result of some kind of modeling of (usually) historical data

that generally includes the application of statistical analysis, operations research, or other

quantitative tool for the purpose of either explaining what is or predicting what will be.

The purpose of the model is to represent critical relationships in such a way to guide

decision makers toward a desired goal. The involvement

and support of

these models

is what differentiates DSS from other kinds of computerized systems. Said differently,

without a model, a system is not a DSS. Hence, to understand DSS, one must understand

models. Unfortunately, in practice, modeling, and especially model management, is the

least developed of the aspects of DSS.

MODELS AND ANALYTICS

Modeling is the simplification of some phenomenon for the purpose of understanding its

behavior. Even before the tsunami of data began hitting organizations, modeling provided

a structure for understanding and predicting events. Modeling simplifies and abstracts

detailed event data to allow understanding of the major forces acting upon the alternatives.

It involves a process of summarizing and accumulating of data. In addition, modeling

involves a process of removing unnecessary detail, thereby allowing the important patterns

to shine through. This is similar to what is illustrated in Figure

4.1.

All of the panels have

Decision Support Systems for Business Intelligence

by Vicki L. Sauter

126

MODEL COMPONENT

Figure 4.1. Process of modeling.

the word "model" in the middle. In the upper panels, you cannot discern the word because

there is too much detail—the important factors are not put together and the unimportant

factors act as noise clouding the image. Slowly, as some factors are accumulated and

the irrelevant noise is removed from the panel (as you move down), it is possible to see

increasing amounts of the word until at the bottom the word is perfectly clear. So it is

with modeling. The key is to identify patterns in the data; one must identify the critical

components and scrape away the others until the important trends are apparent. As you

can see in this diagram, you begin with too much detail to identify any patterns. Once you

scrape away some detail, you begin to see a variety of issues that are not of interest to

your decision situation. Scraping those away, you find a variety of distractions to the core

purpose of your modeling. As those distractions are eliminated, you begin to model. Before

MODELS AND ANALYTICS

127

Figure 4.2. A model airplane.

you get too much clarity, you must first eliminate the unimportant variables. Then finally,

once all of those other issues and the remaining small details are gone, it is possible to see

the model clearly. That is how the modeling process works.

Most people have their first experience with models as children, such as in model

airplane building or model trains. Everyone knows that a model airplane is not a real

airplane and hence will not perform all the functions of a real airplane. However, certain

attributes of the plane are created realistically, such as the number of

wings,

the number of

propellers, the relative size or colors of the plane, and its markings (Figure 4.2). That is,

model makers do not include all of the details of the plane but rather only those that are

important to understand whatever aspects of the plane that are important to the decision

maker. A child might be able to ascertain the development of planes by noting the evolution

of number and placement of wings, the use of propellers, and even how the shape of

the plane has changed over time. Another child, with different interests, might use these

models to learn the colors and markings of planes associated with different countries.

Hence, the amount of detail and the kind of detail necessary for the model airplanes are

dependent upon the interests of the child at that moment. In other words, whether or not

the model is sufficient is dependent upon the needs of the decision maker (in this case,

the child).

Business modeling fulfills the same objective. The purpose of a model is to simplify

the choice context so that decision makers can understand options and their ramifica-

tions clearly. When statisticians develop regression models, their goal is to determine the

factors essential to understanding the variability in the phenomenon of interest. Market

research specialists, for instance, use regression to predict demand for a particular product.

They understand that many factors affect a person's decision whether or not to purchase a

product. However, in developing their marketing campaigns, it is useful to know whether

their product appeals to young, unmarried professionals or to retired blue collar work-

ers,

and whether the desirability of the product is different in different regions of the

country.

Most business decisions have a large number of influential factors, and decision makers

need to filter the essential components of the situation from the irrelevant ones. While it

seems obvious that models fill this need, not everyone feels comfortable with models. It

may not be clear what model is most appropriate. Other times it is clear what kind of

model is needed, but the data are not there to support it. Finally, some decision makers may

128

MODEL COMPONENT

not know how to interpret the results, especially if that means understanding the model's

sensitivity to particular market conditions.

Although models can be applied without DSS, their power is magnified with DSS

because of the inherent flexibility, friendly interfaces, and query capability of

DSS.

Histor-

ically, decision makers needed to rely upon others to develop and interpret models for them

because of the difficulty of running the computer programs associated with models. With

DSS,

decision makers are given personal access to appropriate models and appropriate data

and immediate access to results.

It is this

easy and friendly

access that makes DSS-based models so attractive. Decision

makers can understand the implications of their judgment and modify those judgments

when they appear to be inconsistent with what is known. In addition, because of the speed

and efficiency of analysis, decision makers can examine more alternatives so as to find

a good strategy. Furthermore, the model encourages decision makers to investigate the

variables that are most sensitive to assumptions. Improvement in these aspects of problem

AIDS PLAN Is a DSS resource that allows health care workers in Great Britain to plan re-

sources for HIV/A1DS-related services better The system explicitly encourages decision makers

to focus on

^what-if"

questions so they can creatively experiment with strategies that might

prove useful in meeting the needs of this increasing care-needing group. The DSS can be

used to explore the consequences of alternative strategies or investment

s in resources as well

as the sensitivity of those consequences to particular assumptions about uncontrollable and

unpredictable factors

, This in turn allows decision makers to examine the impacts of the de-

cisions in terms of likely overload, need for further resources, and flexibility to meet future

uncertainties.

Forecasts of demand within particular localities are derived from the COX National Forecasts

by patient categories. Decision makers can elect whether to examine these forecasts at their

low, medium, or high range. This projection of patient demand in turn forms the basis for

experimentatio

n with care options, Costs-of-care options by patient category are used to estimate

the costs and resources required to treat the projected patient demand,

The model's analysis is based on a division of patients into categories that, for planning

purposes, can be considered relatively homogeneous in their demand for services, Criteria that

can be used to classify patients include clinical state, possible drug abuse, age, dependency

,

housing situation, and the presence or absence of informal support at home.

For each category, the health authority needs to identify alternative care options, A care

option is a costed combination of service inputs that constitutes a clinically acceptable method of

treating or supporting a member of the client group. It is defined in terms of the basic resources

needed to supply appropriate care and treatment. Model users can adopt the list of resources

provided with AIDSPLAN or change it to suit their special concerns or circumstances

. Up to 32

different resources can be accommodated in the model, Once users have established such lists of

resources, they can express any given care option as a particular combination of recourses from

the list in specified amounts.

For any particular assumptions made about future demand, A IDS PLAN computes the re-

sources and cost consequences of the identified care strategy. Using a menu, the user can display

summaries of the results at different levels to see the effect of the input assumptions and

to

identify

where further analyses may be needed. In fact, medical personnel currently are using AIDSPLAN

to facilitate discussion of the consequences for services of using AZT prophylactically and the

impact of day care facilities on the provision of inpatient beds.

OPTIONS FOR MODELS

129

analysis in turn aids decision makers in advocacy and implementation of the chosen solution

because they understand more facets of the problem better. For example, the New Zealand

yacht-racing team exploited the benefits of alternative generation and evaluation in its

design of Black Magic 1 and 2, which competed in the America's Cup in 1995. Over

10,000 options were considered during the four-month competition, which allowed the

team to make constant improvements in the design of the yachts at the waterfront facility.

Many believe this systematic evaluation of alternatives led to the remarkable performance

in which the New Zealand team swept the field 5 to 0.

OPTIONS FOR MODELS

A model is a generalized description of a decision environment. The goal of creating it

is to simplify a phenomenon in order to understand its behavior. While that is a nice

definition, it does not help decision makers to understand how to model or even to identify

a model.

Decision support systems can include several types of models, some of which you

have studied in your other classes. For example, statistical models include regression

analyses, analysis of variance, and exponential smoothing. Accounting models include

depreciation methods, budgets, tax

plans,

and cost

analysis.

Personnel models might include

"in-basket" simulations or role playing. Marketing models include advertising strategy

analyses, consumer choice models, and product switch models. The characteristics of these

models differ substantially, as do their uses; each represents simplification of a decision

phenomenon that is useful for understanding some component of behavior. The skills

needed to build and use these models and the kinds of support needed to help less skillful

users utilize the models effectively also differ considerably. Part of the challenge of creating

a DSS is knowing what models need to be included and how they can be supplemented to

make them meaningful and useful for the decision maker.

To determine what kind of model to use, generally we need two kinds of information:

what the decision maker needs and the kinds of data available to use. Since models are

simplifications of real situations that act as vehicles for learning about those situations, we

need to select a model that helps to answer the questions that decision makers pose. Also,

since models have underlying assumptions about the data that are used, we can only select

models for which the assumptions are appropriate for the available data. We will use a

variety of dimensions to describe models and the role they fulfill in decision making, as

shown in Table 4.1.

Table 4.1. Dimensionality

of Models

Representation

Time Dimension

Linearity of

the

Relationship

Deterministic

vs.

Stochastic

Descriptive

vs.

Normative

Causality

vs.

Correlation

Methodology Dimension

MODEL COMPONENT

Representation

The first dimension, the representation, describes the kind of data needed in a model which,

in turn, dictates the necessary approaches used to collect and process the data. In particular,

we are distinguishing between models that rely upon experiential data and those that rely

upon objective data. The difference between the two is the process by which the model is

generated, not the answer that is derived.

Experiential models rely upon the preparation and information processing of peo-

ple,

either individually or as a group. These models might include judgments, expert

opinions, and subjective estimates. For example, diagnostic software used by physi-

cians to help in prescribing treatment for tumors or blood diseases models the expe-

rience of expert practitioners. Similarly, a forensic animation simulation was created

to convict a Florida man of vehicular homicide. The simulation showed how his truck

drove into a group of children (one of whom was killed) and then left the scene of the

accident.

One of

the

problems associated with the use of such models is their subjectivity in use.

In such modeling, the information used and the manner in which it is used to make a choice

are up to the decision maker. If two individuals attempt use the same behavioral model, they

may come to different conclusions because they are drawing upon different experiences

and are likely to weight those experiences differently. In the case of

the

forensic simulation,

the verdict was appealed on the basis of the use of the simulation which, according to

the defense, misrepresented the scene of the accident (which happened at night) and the

automobile.

Objective models, on the other hand, rely upon specified, detached data and its analysis

by known techniques. They are considered "objective" because the data considered and

the way they are used are specified, constant, and independent of the specific decision

maker's experiences. Consider the Advanced Trading System from Scottrade shown in

Figure

4.3.

This system allows decision makers to access real-time stock quotes, historical

data, and models for analyzing the data. The return on investment computed by one user

for a particular option will be the same as the return on investment computed by another

user for that same option. Hence, there is no subjectivity associated with the analysis.

However, that in no way means that they are unbiased or lead everyone to the same

conclusion. Clearly, we can bias results by the selection of the variable, time period, or sam-

ple group. For example, conclusions about the yield of investments can vary substantially

by the time horizon considered; stock market investments tend to provide poor yields when

examined over short time horizons but excellent yields, on average, when examined over

multiple decades. Both provide an "objective" view of the performance of a portfolio, yet

they provide very different conclusions; not providing both views presents a biased view

of the problem. The ability to recognize such biases and thereby study multiple aspects of

a problem is one of the advantages of using a DSS.

Neither the experiential nor the objective model is appropriate all the

time,

and each has

its own strengths and weaknesses. Objective models have the advantage of being straightfor-

ward to apply and easily replicated with new data. In addition, they can save time in that they

do not require the establishment of extensive experience such as is needed for some forms of

behavioral modeling. These models have limitations as well. The basic assumption underly-

ing objective modeling is that the simplification of reality necessary to create

a

mathematical

model does not eliminate the essential issues controlling the decision environment. That

is,

it

assumes that

the

most important factors, such

as

competition, regulation,

prices,

and technol-

ogy, are represented in the simplified model in a manner similar to that in the actual decision

OPTIONS FOR MODELS

131

Figure 4.3. Screenshot from Scottrade's Advanced Trading System Software. The image is reprinted here with the

permission of Scottrade.

environment. If these factors change in a significant

way,

the mathematical models would not

be appropriate because the essence of the decision environment and its probable reactions

would not be represented. Under these circumstances, it is important to rely on experiential

models.

Some DSS allow for the integration of both the objective and experiential models. For

example, the DSS facilitating the U.S. Army plans for future needs of materiel incorpo-

rates both kinds of modeling. Objective models are built based upon quantitative analysis

of historical data. In this case, the historical data represent past demands for and uses of

the materiel over time. The projections combine models that first assume a continuation

of past patters of materiel use and then take into account planned activities such as ma-

jor exercises. These forecasts are supplemented with heuristics about possible changes in

the needs during the upcoming time horizon; expert opinions and human judgment are

included to alter the projections. The DSS helps the user evaluate the combined model per-

formance by continually measuring trends and alerting the decision maker to changes in the

trends.

132

MODEL COMPONENT

Time Dimension

The time dimension identifies how much of the activity of the decision environment is

being considered. The two ends of the continuum are static models and dynamic models.

At the static end, models represent a snapshot in time of all factors affecting the decision

environment. Such models assume that everything will remain the same. Similarly, such

models assume that there is no dependence of later decisions or actions on the choice under

consideration. Dynamic models, on the other hand, consider the decision environment over

some specified time period. They may consider the same phenomenon during different

periods of time or interrelated decisions that will be considered during different time

periods.

Time can be represented in models in a variety of

ways.

An example of

its

explicit use

of time to examine some phenomenon is shown in Chapter 2. The software in use at Gap

Minder (example results of which are shown in Figure 4.4) looks at how the factors under

consideration change by increments of

a

year. In this way decision makers can examine the

Design Insights

Modeling Chip Architectur

e

Designing chip architecture for the best performance and smallest size is an exceedingly difficult

task. Today, computers solve the problem by considering possible combinations. They are fast, but

the computer lacks both intuition and visual pattern recognition

. These are not only characteristic

s

at which humans excel but also characteristics that could yield a better or even optimal design.

Researchers at the University of Michigan are developing mechanisms to combine the speed of

computers and the skill of humans in a project called FunSAX By solving problems using the

Fun SAT board, players contribut

e to the design of complex computer systems. Although the

humans believe they are just selecting actions that will turn all buttons green, they are in fact

solving complex problems or selecting the best arrangemen

t of options. The solution is then given

to a computer scientist who translates that solution into hardware design. The researchers hope

to use this combination of objective and subjective modeling to improve chip designs, databases,

and even robotics. Perhaps someday similar

l

'games" can be used to improve other decisions,

Adapted from

Dc

Orio,

A.

and

V.

Bertac^o

,

Design Automation Conference

(DAC),

San Francesco,

CA,

avail

able

at;

http ://w w w,

eecs.

um ich. edu/—va per ia/rcscarch/publicaüons/DAC09Fun STA .pdf July

2009.

Used

with

permission of Mr

r

De

Οτίο and

Dr,

Bertacco.

The Fun SAT

1+

game"

is

available

at

http://funsaLeecs.unieh.edu.

OPTIONS FOR MODELS

133

Figure 4.4. Looking at intervals of time for patterns. Gapminder software shows incremental

changes in the graph on an annual basis. In this way, decision makers can examine the relation-

ship's changing nature over time. Visualization from Gapminder World, powered by Trendalyzer

from www.gapminder.org.

134

MODEL COMPONENT

directionality of the change, the times at which the magnitude of change shifted direction,

and the relative change of

a

variety of observations. Other ways of representing time include

using time as a variable in the model, examination of results in a "before" and "after" time

period, and using models that use interdependence of time periods explicitly, such as with

dynamic programming.

Linearity of the Relationship

This third factor of a model is called linearity. It refers to the relationship between two or

more factors. Such relationships are either linear or nonlinear. Everyone has seen a linear

relationship in two dimensions; it is expressed by a straight line. It can be interpreted easily

as the more of

x,

the more of v. For example, the larger the warehouse, the greater the

storage volume available.

Anything other than the straight line is referred to as a nonlinear relationship. The

two-dimensional graphs in Figure 4.5 and the three-dimensional graph in Figure 4.6 are

nonlinear. Nonlinear relationships require the user to specify the kind of relationship be-

tween and among the variables. For example, sales related to the natural log of advertising

expenditures, or sales related to the square root of price, or sales related to the square of

time spent with a sales representative are all nonlinear relationships. Such relationships

do not have the nice intuitive interpretation of linear models. Nor is it obvious how to

build the model. The linearity, or lack

thereof,

affects the kind of model that one can use.

For most linear model solution techniques there are parallel nonlinear solution techniques.

The nonlinear models are more complex. The first—and hardest step of the process—is

to specify the nature of the relationship. While it may be easy to determine the kind of

relationships shown in Figure 4.5, if you have some mathematical background, data rarely

Figure 4.5. Nonlinear relationships.

OPTIONS FOR MODELS

135

Figure 4.6. Nonlinear higher dimension relationship.

come so well behaved. Generally they include error terms such as that shown in Figure 4.7.

Clearly this makes it harder to determine exactly what nonlinear function should be used

with the data. It takes time, patience, experience,

and

an understanding of the phenomenon

being modeled to get it right.

For this reason, it is tempting just to use the linear model to approximate the nonlinear

data. Not only does that avoid the problem of having to determine the underlying function,

but also the linear models are better behaved, easier and faster to solve, and generally have a

straightforward approach to solution. There are times when such approximations are good

enough, especially since the techniques of nonlinear models generally are harder and slower

to solve, require a hierarchical approach, and often result in "good answers" rather than

"the best answers." Other times, however, the conclusions gained from the linear model are

inappropriate for the nonlinear world.

Deterministic Versus Stochastic

Most of the modeling taught in business colleges is deterministic. For example, consider

linear regression. You might want to predict sales using price and advertising. To do this,

you collect past data about the three variables and run the regression. You might result with

something like

Sales,

= 5.64 + 16.1 Price, + 0.58 Advertising,

136

MODEL COMPONENT

Figure 4.7. Nonlinearity with randomness.

To use the model, a decision maker would substitute in values for price and advertising

and from it would get an expected value for sales. This is deterministic in that it uses fixed

variables that are determined by averaging the error terms over the training data set. It is a

method that works well for many situations.

Stochastic modeling, on the other hand, explicitly uses probabilistic distributions for

one or more variables in the model to view how situations might evolve over time. These

models use historical data, but rather than the specific variable associations (as shown with

the regression above), stochastic models use the fluctuations in the data

to

determine a likely

underlying probability distribution for one or more of

the

variables. Using those underlying

distributions, a model is constructed to reflect the scenarios, decision points, and outside

influences on the system. The model then is run hundreds or thousands of

times

so decision

makers can view the range of the impact as well as specific estimates. The most common

form of stochastic modeling is based on Monte Carlo analysis. As you can see from Figure

4.8,

the result of such an analysis is the outcome of some variable, say production rates,

associated with each different combination of randomly generated parameters. We then

look for the average or typical value (the middle line) and for the typical range of values

(between the outer

lines).

The questions we might answer is whether these values are "good

enough" or perhaps how we can improve the values by changing things (such as adding

another line in the production facility).

Descriptive Versus Normative

Another choice to make selecting a model is whether you wish it to be descriptive or

normative. Descriptive models are those that report what is happening in the data. It might

be the sales of widgets by division, the profitability of a sales

line,

the absenteeism associated

OPTIONS FOR MODELS

137

Figure 4.8. Results of a Monte Carlo analysis.

with

a

particular facility, or the number of radio advertisements run by the competitor. These

descriptive models might be created to provide decision makers with a quantitative view

of what is happening in the organization or part of an organization as background or for

monitoring. Or, the descriptive models might serve as predictive analytics, which attempt

to forecast how factors such as sales, profitability, absenteeism, or competitor's ads will

behave in the future. As said earlier, of course, such models are only valid if the factos

pushing on the phenomena are the same in the future as they have been in the past.

The alternative to descriptive models is normative models. These models represent

an

ideal value

of sales, profitability, or absenteeism in an organization. The output of the

normative models, perhaps of sales, is then compared to the actual sales to determine if

operations are running as we expect they should. This form of modeling does not provide

a view of how the organization is changing, that is, how sales are growing (or declining)

over time, but rather simply a view of how the current organization is competing relative

to a set of standards or values.

Causality Versus Correlation

The relationship between correlation and causality is one of the most misunderstood and

misapplied in all of modeling. Correlation, however it is represented, refers to the amount

and direction that two or more variables vary together. It might be thought of

as

the level at

which the variables simultaneously change. If two variables move positively together, that

MODEL COMPONENT

means as one increases, so does the other; if the correlation is negative, the variables move

in opposite directions. Similarly, the magnitude of the correlation indicates how similar

the movement is a larger correlation means the rate of change of the two variables is more

similar.

However, correlation does

not

say anything about what caused this association. For

example, there is a positive correlation between education and income. The fact that

education and income are correlated does not imply that getting more education causes

your income to increase. It is possible that people from higher incomes simply get more

education. Or, it is possible that there is another factor, say intelligence, that causes changes

in both variables. Similarly, increasing price does not cause a drop in sales. It may be that

reduced sales causes a company to increase its price to cover its costs. Or, it may mean that

a competitor is pushing both variables to change. If the goal in the analysis is to determine

what

causes

changes in some factor, then in addition to correlation, it is necessary to prove

that it is impossible that anything else but one factor could have caused the change in the

other factor. This requires the design of a scientific experiment that controls the variables

to approximate such a counterfactual state of the world. Generally this is achieved by

conducting experiments on identical items or randomizing exposure to the experimental

factors.

Methodology Dimension

The last dimension, methodology, addresses how the data (whether objective or experien-

tial) will be collected and processed. There are five general methodologies: (a) complete

enumeration, (b) algorithmic, (c) heuristic, (d) simulations, and (e) analytical. In complete

enumeration, by far the hardest and most expensive option, information about

all

feasible

options is collected and evaluated. Under many circumstances, complete enumeration

is totally impractical. However, there are some contexts for which it is necessary or

desirable. For example, the U.S. Census is an example of complete enumeration in which

all individuals in the United States are identified and

counted}

The purpose of counting

all individuals is to understand the population shifts in the United States so representation

in the Congress can reflect actual population density. Rather than sampling various areas

in each state, the government identifies every person individually.

Complete enumeration also has been useful in the application of neural networks

of transaction files for pattern recognition. For example, a neural network system was

constructed for Mellon Bank of Chicago to identify suspicious credit card activity that

might be indicative of stolen credit cards. Historically, both human auditors and electronic

expert systems identified dubious transactions through abrupt increases in either the number

or the size of transactions. By examining all the transactions, the neural network identified

a change in

small

purchases as an indicator of stolen credit cards. In fact, at that time, card

thieves were using small purchases, often as little as $1, in pay-at-the-pump gas stations,

to determine whether the cards were still being accepted. It was this complete enumeration

of transactions, supplemented by pattern recognition capabilities, that allowed the system

to respond quickly to the presence of criminal behavior.

The second approach, the algorithmic model, is the development of a set of proce-

dures that can be repeated and will, eventually, define the desired characteristics of the

1

It

has been noted

that

the

U.S.

Census process does

not count

homeless

individuals and underestimates

their

numbers.

Strictly speaking, then, the census is not

a

complete enumeration.

OPTIONS FOR MODELS

decision environment. Such models are best represented by the field of operations re-

search/management science. Algorithms have a set of repetitive calculations that can be

implemented to find the best answer. The set of calculations itself

is

based upon the charac-

teristics of

a

particular

problem.

Unlike total enumeration, an algorithm identifies promising

information that can be used to identify the best outcome without first evaluating all pos-

sible options. An example of such a modeling technique is the Simplex Algorithm. To use

this model, we need to represent a problem as a linear program, determining an objective

function that can be optimized (either maximized or minimized) and a set of constraints.

Typically the objective function uses the minimization of

costs,

the maximization of utility,

or some related concept. The constraints define the availability of scarce resources such

as time, money, and inputs. If we can represent the problem as a linear program, we can

use repetitive operations based upon matrix row reduction calculations and find the best

solution to the problem.

2

These repetitive operations are simple arithmetic operations; the

process of applying them is the algorithm.

Algorithms are used widely today in business, organizations, and government. They

can help decision makers know how to place investments, where to advertise products, or

how to assign staff to projects. One area where algorithms are used heavily is in personnel

planning and scheduling. For example, many hospital systems use algorithms to assign

nurses and other staff to shifts. In some cases, the systems include measures of "intensity"

of patient illnesses so that they can determine whether the optimal general staffing levels

will meet the specific needs on a daily basis. Similarly, the U.S. Army uses an algorithm-

based DSS called

ELIM-COMPLIP

with input from other modeling forecasting systems

to plan for deployment of personnel to various tasks so as to meet their strength needs as

specified in the Force Structure Allowance.

The third possible model process is heuristic. Generally heuristics are applied to large

or ill-structured problems that cannot be solved algorithmically. The goal is to find a

satisfactory solution that is reasonably close to optimal. All heuristics involve searching,

evaluating, learning, and more searching to find a good solution. They are usually developed

for a particular problem in order to take advantage of the structure of a problem. Some

heuristics are designed to construct solutions; others are designed to improve existing

solutions. Since heuristics are so dependent upon a particular representation of a problem,

they are not often generalizable to other problems.

Heuristics can be quantitative solutions to a problem or behavioral solutions to a

problem. In the former

case,

the model is a numeric representation of

a

choice and we focus

on numeric processing. Typically, a quantitative heuristic is developed as an alternative

to using a quantitative algorithmic approach, if, for example, a reliable algorithm is not

available, if

the

computation time is excessive, if

the

data are limited, or if

the

problem is so

big it cannot be reasonably simplified otherwise. For example, if the decision variables in

a problem are restricted to dichotomous (0-1) values or integer values, known algorithms

may fail to find an optimal solution. This might include a firm's assignment of production

processes

to

particular production facilities or

a

financial

institution's assignment of deposits

to lockboxes. Similarly, if the objective to a problem is nonlinear, or if there are many

variables or constraints, known algorithms may fail to find an optimal solution. Some

heuristics can be identified that take advantage of the mathematical structure of a problem

to find good answers to these problems.

2

There are some special problem structures that cannot be solved using this algorithm. In addition,

some problems cannot be solved

practically

with this technique because the number of variables

and/or constraints is so large it would take

a

prohibitively long amount of

time

to solve the problem.

140 MODEL COMPONENT

Modeling Insights

Linear Programming

To understand algorithms and their use, let us consider a specific problem. An MIS Club plans to

sell two special fruit baskets for the upcoming holiday season. Fruit basket A contains 3 apples, 4

oranges, and I honeydew melon and sells for

$8,

Fruit basket

B

contains 4 apples, 3 oranges, and

2 honeydew melons and sells for $12. The amount of each fruit available and their costs to the

MIS Club are shown in the table below. If it is assume

d that the MIS Club can sell all the baskets

it makes, how many of each one should they make?

Quantity

Available

Cost per

Piece

Apple

Orange

Melon

160

180

60

$030

$0.20

$L20

The first step is to represent the problem mathematically

. In this case, we will have two

variables, x and y, where

x

represent

s the number of fruit basket A to make and

y

represents the

number of fruit basket B to make. We know that each fruit basket A sells for $8 and each fruit

basket B sells for $12, but in order to know how much profit wc will make, wc must compute the

costs of each basket. Basket A contains 3 apples at

S

,30,4 oranges at $ .20, and I melon at SI ,20,

so it costs $2.90 to make up the basket (if we assume the actual basket is free)· Hence, the net

profit from basket A is $5.10. Using a similar method, we can find that the net profit from Basket

B is $7.80. Hence, our objective is to:

Maximize 5-10X +7.80y

However, there are constraints dictating the availability of fruits which must be met. Using the

quantities above, they are:

Apples

Oranges

Melons

3x-h4>< 160

Ax

+ 3y < 180

\x-\-2y

<60

Conceptually, the algorithm for solving this problem looks at possible values for

x

and

y

and

selects the one that maximizes our objective. Consider the graph below;

80

- Oranges constraint

Apples constraint

Melon constraint

40 60 80

Number of fruit basket A

OPTIONS FOR MODELS

141

If the heuristic is behavioral, then we consider the relationships between concepts and

use symbolic processing of the data. In fact, this kind of behavioral heuristic is generally

referred to as expert systems (a branch of artificial intelligence). Expert systems use rules,

frames, objects, and metarules (often referred to as demons

3

) to replicate the solution

3

The term "demon" in

a

programming environment refers to

a

portion of

code

that lies dormant until

a particular event, such as the change in the value of a variable, causes the code to process. These

demons might cause particular actions to occur, such as the searching of a database, or they might

prohibit actions to occur and to take the user along a different path of

code.

The algorithm

"knows"

to look for the feasible combinations of the two types of fruit baskets,

as shaded in the-graph. Further it "knows" that the best combination is going to be one of the

four "extreme" or comer points highlighted above. The algorithm evaluate

s an extreme point

with regard to the objective

(5.1 Ox

-h 7.80;y). It then looks at the adjacent corners to determin

e

if one of them give a better solution. If so, the algorithm moves to that new point and begins

again. In essence, the algorithm moves from comer to corner, always improving the value of the

objective. With large problems, the process is important because one can have many variables and

many constraints resulting in millions of corner points. Since the algorithm follows a systematic

approach to improvement, it ends up checking only a small percentage of the possible points. In

this case, it is the combination of 36 fruit baskets of type A and 12 fruit baskets of type B, giving

a profit of $277

+

20 to the MIS Club.

DSS in Action

MLB Schedules as Models

Baseball is called the "great American pasttime" because so many Americans share a passion

for the game. The game may live or die by the pitcher or the next power batter, but the schedule

is dependen

t on modeling, The Sports Scheduling Group (556) uses mathematical programming

and high-performance computers running virtually nonstop for months to develop a schedule for

major league baseball. According to one of the partners of SSG, "a typical model for a sports

scheduling problem is a combinatoria

l design with nasty side constraints and multi-objectives.'*

Schedule makers deal with conflicting requirement

s and preferences as a matter of course,

but as the financial and competitive stakes in athletics rise, so does the complexity of creating a

balanced schedule

. To maximize revenue, it is crucial to have important games televised on the

right days and times. These requirements frequently conflict with more traditional requirement

s

of a 'Tair" schedule that balances strength of schedule, home and away games, and travel.

SSG must consider the following constraints when developing a schedule:

* Each club plays 162 games and 52 series, including 13 at home on weekends.

* Games within each month and during summer dates should be reasonably balanced

between teams.

* Single-series and four-series home stands and road trips should be minimized; two- and

three-series home stands and road trips are preferred.

* No more than four series home stands or road trips should be scheduled,

* There should be no doubleheaders in the original schedule

.

* Considerations must be made to the miles traveled by one team during a season. No team

should travel in excess of 50,000 miles over the course of the season,

* Three game series are optimal (minimize number of

two-

or four-game series),

In addition, SSG entertains the requests of the teams, the television network

s broadcasting the

games, and the MLB Players Union*

142

MODEL COMPONENT

technique that an expert would use to solve an ill-structured, nonquantifiable problem.

These models can give meaning and context to the symbol and incorporate subjective

information about the validity of an answer or the way in which the answer should be used

to obtain a solution.

The fourth approach to modeling is simulation. Unlike algorithmic and heuristic mod-

eling, which provide a normative answer, simulation provides descriptive results. The

goal of simulation is to imitate reality either quantitatively or behaviorally. Typically, this

Modeling Insights

Presidential Selection Heuristic

s

Every four years there is a great deal of money spent on trying to predict who will win the

U.S.

presidentia

l election. Pundits examine the various segments of the population carefully and

determine the issues that are most important for each group, who best addresses those issues (for

the groups)

, and what the likelihood of that group voting will be. There arc millions of dollars

spent to predict who is likely to win the election

* As the viewing public knows, there are many

flaws to these predictions.

Allan J. Lie htm an, professor of history at The American University in Washington, D.C.,

looks at the situation in a different way. He applied statistical pattern recognition algorithm from

seismology to the question of who would be elected. Professor Lichtman began with nearly 200

questions, which were all binary (yes-or-no) variables, and the algorithm picked those which

displayed the greatest difference between the proportion of the time the variable was "yes"

for years when the incumbent party won and the corresponding proportion for years when the

challenging party won using all

U*S*

elections starting with 1860 as the training set. Over lime,

he narrowed it to

13

keys. They are:

1.

The incumbent party holds more seats in the ILS, House of Representative

s after the

midterm election than after the preceding midterm election.

2

+

There is no serious contest for the incumbent-part

y nomination.

3,

The incumbent-part

y candidate is the current president

4,

There is no significant third-party or independent candidacy.

5,

The economy is not in recession during the campaign.

6, Real (constant-dollar) pcr-capita economic growth during the term equals or exceeds

mean growth for the precedin

g two terms.

7,

The administratio

n has effected major policy changes during the term.

8, There has been no major social unrest during the term,

9, The incumben

t administratio

n is untainted by major scandal.

10.

There has been no major military or foreign-pol icy failure during the term.

11.

There has been a major military or foreign-policy success during the lerm.

12.

The incumbent is charismatic or is a national hero.

13.

The challenger is not charismatic and is not a national hero.

According to Dr. Liehtman's models, if six or more of these statements are false, the incumbent

party loses the popular vote. Using that criterion, the model has only been wrong twice, in 1876

and 1888. Of course, in the United States, it is the electoral vote, not die popular vote, that

determines the winner, so sometimes this method does not predict who will actually be in the

White House.

Samulson, D. "Road to the White House;'

ORMS

Today,

Vol 35, No 5, October 2008. This material is

reprinted with permission of

the

publisher and the author.

OPTIONS FOR MODELS 143

Negotiation Ninjas, developed by researchers at Southampto

n University, are intelligent agents

that use heuristics to help bring together buyers and sellers on the shopping website Aroxo, The

agents use a series of simple rules—known as heuristics—to find the optimal price tor both

buyer and seller. The heuristic

s guide not only the price but also the ways to address multiple

simultaneous negotiations

. Sellers must answer a series of questions about how much of

a

discount

they are prepared to offer, whether they are prepared to go lower after a certain number of sales

or at a certain time of day, and how eager they are to make a sale. Buyers only need to identify

the item they wish to purchase and the price they are willing to pay for it, The agents then act

as an intermediary, scouring the lists of sellers who are programmed to accept a price in the

region of the one offered

. If they find a match, the seller is prompted to automatically reply with

a personalized offer. The buyer then has a choice to accept, reject, or negotiate. If they choose to

negotiate, the agent analyzes the seller's criteria to see if they can make a better offer. The process

continues until either there is a sale or one of the parties pulls out.

One system using nonquantitative heuristic

s is

PROSPECTOR.

The purpose of this system is to

predict mineral deposits given geological information about a region. Some of

PROSPECTORS

rules arc the following,

* RULE 1: IF the igneous rocks in the region have a fine to medium grain size, THEN they

have a porphyritic texture (0,5)«

* RULE

2:

IF the igneous rocks in the region have a fine to medium grain size, THEN they

have a texture suggestive of a hypabyssal regional environment (2,

(λΟΟΟΟΟ

I).

* RULE 3; IF the igneous rocks in the region have a fine to medium grain size and they

have a porphyriti

c texture, THEN they have a texture suggestive of a hypabyssal regional

environment (100, 0.0000001).

* RULE 4: IF the igneous rocks in the region have a texture suggestive of a hypabyssal

regional environment, THEN the region is a hypabyssal regional environment (65, 0.01).

* RULE 5: IF the igneous rocks in the region have a morphology suggestive of a hy-

pabyssal regional environment

, THEN the region is a hypabyssal regional environment

(300,ΟΌ001),

* RULE 6: IF the region is a hypabyssal regional environment, THEN the region has a

favorable level of erosion (200, 0.0002).

* RULE 7; IF Coeval volcanic rocks are present in the region, THEN the region has a

favorable level of erosion (800, I).

The system processes these and other rules much the way an expert geologist would to examine

the geological, geophysical, and geochemical, data to predict where ore-grade minerals could be

found. The numbers in parentheses indicate measures of certainty with the conclusions that are

built into the reasoning process.

Source:

Waterman, D. A. (1986)

A Guide to Expert Systems,

"Prospector Rules,

TT

p. 58. Reproduced with

permission of Pearson Education. Inc.

144

MODEL COMPONENT

Figure 4.9. Simulation with animation. (Source: The Great Flu, Erasmus University, available: http://thegreatflu.

com.) Application was developed jointly by Erasmus University Medical Center and the Ranj. Serious Games. Image is

reproduced with permission.

involves the repetition of an experiment and the description of the characteristics of certain

variables over time. For example, a simulation of a factory would include a variable that

measures the amount of time an average part spends waiting in lines and the amount of

time it takes to process the inventory. Using the mathematics underlying the simulation, we

could vary the demand for products, the raw material arrivals, and the number and types

of production lines and study the impact of these variations on the amount of time one part

spends waiting in line and making a transaction. With today's simulation software, decision

makers can vary decision variables and see the impact with animation.

Consider the simulation shown in Figure 4.9. This simulation was created by influenza

researchers at Erasmus Medical University Center in Rotterdam to help decision makers

examine the activities associated with fighting a pandemic influenza outbreak. Once the

simulation has started, decision makers have a variety of actions they can take in each region

ranging from improving research facilities, to stockpiling vaccines and antiviral medicines,

to isolating sick individuals; closing schools, markets, and airports; or simply stating

warnings. Each activity costs money, and the decision maker is given a budget. During

the simulation, decision makers can view information about the spread of the virus across

the world and the number of resulting deaths. Through use of such simulation, decision

makers can experiment with various strategies and gauge their effectiveness without putting

a single person in jeopardy of the illness.

Simulations help decision makers understand how external influences can affect the

outcome of their decision. For example, computer companies rely heavily upon simulation

in deciding when to introduce new models. Simulations model customer demand, pricing,

and dealer inventories and simulate a variety of relevant conditions, such as component

OPTIONS FOR MODELS

145

price changes or even the impact of a rival model. In this way, the managers can evaluate

the risk

before

taking the risk.

Similarly, personnel departments use "in-basket" simulation exercises to help indi-

vidual managers determine the best approaches to addressing the problems that arise in

managing people. In this case, the manager measures not a mathematical variable, but

The U.S. military is one of the most significant users of simulations in the world today. The

Generalized Air Mobility

Model,

or GAMM, simulates the entire theater airlift system's movement

of cargo from source to destination. Hence, the DSS provides simulation of flights, airdrops,

overland cargo transshipment, and survivability of cargo in the various modes of transportation

.

(The DSS does

not

simulate the outcome of the campaign, just the ability of the airlift system to

meet the operational demands of a given scenario.)

The quality of the insight from this simulation, as in any simulation, comes from the quality

of the measures that were built into the system for evaluation. Historically, the military used

measures such as rate-of-cargo movement, average aircraft flying time per day, utilization rate,

and departure reliability While these measures provide some indication of the basic throughput

of the operation, they do not measure the effectiveness of the mission or how it supports combat

forces. Hence, GAMM has factors of evaluation such as:

• Timeliness of deliveries

• Effectiveness in making multifligh

t deliveries within narrow time and location constraints

such as those necessary for combat mission

s

• Ability to move large, oversize items

In addition to providing operational logistics for a particular campaign, GAMM also can

predict where long-term airlift characteristics need to be changed and hence offer insights into

fum re designs.

The costs of providing health care have skyrocketed over the last 20 years. At

the

same time the

incidence of infections, especially antibiotic-resistant infections, contracted during hospitalization

has increased significantly

. States have recognized the impact of these secondary (not existing

upon admission) infection

s on health cane costs, and some have introduced legislation to reduce

payments to hospitals with high rates of secondary infection. Clearly it is in everyone's best

interest to reduce the incidence of infections contracted during hospitalization

. But, this is a

difficult problem to solve due to interactions among the various pathogens, categories of illness

of the patients, and occupancy rate of the hospital. In other words, it is hard to know "where to

start." However, researchers worked with Cook County Hospital in Chicago to build simulations to

represent various scenarios of these variables so they could study the relative efficacy of improve

d

hand-hygiene protocols versus changes in patient isolation policies. They found both policies

could have a significant impact on the rate of infections. However, when they also examined

the costs—both to the patient and to the hospital—under various conditions, they determined

that improved hand-hygiene protocols were more appropriate as a first approach to solving the

problem. Further, the researchers provided insights into condition

s where the policies should be

changed and what the associated costs would be.

146

MODEL COMPONENT

rather the reaction of another individual in order to experiment with more positive and more

negative reactions and determine which will provide the desired effect. Finally, today's

technology can make it possible to simulate how it feels to drive a given automobile over a

variety of surfaces and in a variety of conditions to determine which car provides the most

desirable ride given its cost.

The essence of constructing simulation models is to simplify the elementary relation-

ships and interdependencies of the situation being considered. While it does simplify the

conditions, simulation also allows us to build in real-life complexities that might affect the

variables being measured. It is descriptive in its answer, thereby encouraging

"what-if"

kinds of experimentation in which many alternatives can be considered independently, and

time is compressed so that long-term effects can be measured quickly.

Design Insights

Modeling Failures

Computer simulations are not replicas of reality. For example, Boeing Co. Engineers used simula-

tion to design a fuse pin that held the engines to the wing for its 747 cargo plane. After El Al Israel

Airlines had a crash in 1992, where the plane killed over 40 people in the Netherlands, engineers

reviewed their simulation. They found that the simulation had missed several weak points in the

design of the fuse pin. The fuse pin had in fact broken, causing the crash.

odeling Insights

Finding bin Laden

Professors in the Geography Department at UCLA applied biogeographic models to the question

of locating Osama bin Laden in the spring of

2009.

Biogeographi

c models use known properties

of plants and animals to predict how they will distribute themselves over space and time. These

models were applied to publicly available satellite imagery.

The particular models employed are called a

"

'distance

decay theory" and "island biogeog-

raphy theory/' They were employed because they are associated with the distribution of life and

extinction. Distance decay theory states that as one goes further away from a precise location,

there is an exponential decline in the turnover of species and a lower probability of finding the

same composition of

species.

The theory of island biogeography states that large and close islands

will have higher immigration rates and support more species with lower extinction rates than small

isolated islands,

These theories can be applied over varying spatial scales to posit bin Laden \s location based

on his last reputed geographic location. Distance decay theory would predict that he is closest to

the point where he was last reported and, by extension, within a region that has a similar physical

environment and cultural composition (that is, similar religious and political beliefs). For instance,

the further he moves from his last reported location into the more secular parts of Pakistan or

into India, the greater the probability that he will find himself in different cultural surroundings,

thereby increasing the probabilit

y of his being captured or eliminated. Island biogeographic theory

predicts that bin Laden is in a larger town rather than a smaller and more isolated town where the

extinction rate would be higher. Finally, high-resolutio

n analyses of a city can be undertaken to

identify individual buildings that match bin Laden's life history characteristics. For example, he

reportedl

y has a small entourage of body guards, requiring a structure that contains at least three

rooms.

Using these methods, the biogeographers identified not only a specific town in Pakistan in

which bin Laden is likely to be located but also three specific buildings in which he is likely to be

located. However, no national security agency has commented on whether they have applied this

methodology or whether or not the professors were accurate.

PROBLEMS OF MODELS

Simulations are not without their disadvantages, however. They do not provide an

optimal solution; instead they provide information about conditions from which we can

glean a good or possibly optimal solution. Like heuristics, inferences are not transferable

beyond the specific type of problem being considered. Finally, and most important, the

construction of simulations can be slow and costly.

The last type of methodology is the analytical model. Analytical modeling refers to

the process of breaking up a whole into its parts and the associated process of examin-

ing the parts to determine their nature, proportion, function, and interrelationships. Where

phenomena are well defined, analytical approaches solve for related variables that have

specified properties within limits. For example, the phenomenon of gravity is well de-

fined so that we can use specified equations to describe how an object will fall. Where

phenomena are not well defined, which includes virtually all business-related phenomena,

the analytical approach determines how to separate a given problem into its constituent

parts and determine what subcomponents are most important in affecting the interactions

with other subcomponents. Statistical analyses, especially regression and other predictive

models, provide good examples of analytical modeling.

Consider, for example, the process of creating strategies for football games. The

interdependence of the players and the complexity of the plays make it difficult for any

individual to make choices without help. National Football League teams use DSS with

sophisticated analyses to make these decisions. The DSS helps the coach to understand the

tendencies of

his

own team and the opposition and hence to plan strategies that will respond

to them. The New England Patriots use a DSS to select the best players at the lowest cost to

decide what play to run or whether to challenge a referee's ruling and even how to improve

total fan experience.

PROBLEMS OF MODELS

Modeling is not without its problems. Modeling depends on understanding the factors that

impact the phenomenon of interest and using those variables in the correct proportion. The

failure to identify an important variable, to select an inappropriate time horizon, or

to

overfit

the model to some time period will decrease the value of the model to the decision makers.

Quantitative modeling, in addition, assumes that the factors acting upon the phenomenon

will continue to be important and will continue to work in the same fashion as in the past.

For example, most public transportation companies have models to predict ridership. They

use the models to decide routes for buses and trains and how often to schedule vehicles

on each route. If done well, the models provide a good mechanism for planning. However,

when gasoline prices suddenly surge, the assumptions about ridership change significantly

and the models no longer provide a reliable output for decision making. The use of models

assumes the underlying assumptions continue to be true. Decision makers need to consider

if that is true.

Not knowing if

the

assumptions are true is one problem. Knowing the assumptions are

not true and continuing to use the models make their use more hazardous. Consider the

financial institutions and their use of models prior to the recession of

2009.

Analysts chose

to program their risk management systems with overly optimistic assumptions and to feed

them oversimplified data. In other words, financial analysts modeled the system so as not

to identify all of the risks and perhaps maybe even the correct risks. Rather than noting

recent volatility in the market, the models looked at several years of trading history, which

dampened the impact of an impending crisis. Others, it is claimed, developed models that

did not reflect the complexity of the financial products being traded.

MODEL COMPONENT

DATA MINING

One kind of modeling that is particularly important in DSS is data mining. When we think

of mining, we think about digging deeply into some repository to find something of value.

When one mines for diamonds, one digs into seemingly common rocks to find brilliant

pieces of carbon. Said differently, one needs to look carefully through vast repositories of

useless rock to find that one nugget that is valuable. A similar process is used for data

mining. Data mining might easily be defined as the process of extracting valuable patterns

from a mass of data. Companies often mine the data to find evidence of theft or fraud,

patterns of purchasing (or other behavior of interest), or evidence of the need for new

products, new markets, or new sources of

revenue.

This is not a new idea; companies have

been trying to mine their data for hundreds of years. What

is

new is that companies are

able to collect and save much more data now than ever before. Similarly, although there

have been many data-mining tools available for some time, today's processing power has

brought us new tools that increase our ability to find patterns in the data.

Consider, for example, one of the largest procurers of data in the world today, Google.

Every day, there are several million searches on Google to find anything from a product

for a gift to health information. Google saves the data. It is not just the search, but if you

have logged in, it saves your name and email address, the date and time of day, and your

Internet Protocol (IP) address. The IP address, of course, gives Google information about

how you are connected to the Internet and the country (and, in the U.S., the city and state)

from which you connected to the Internet. There have been hundreds of billions of searches

since 1997, when the search engine was launched.

Google mines its data, meaning that it attempts to find patterns in the searches that are

useful. One such mining exercise is the Google attempt to predict influenza outbreaks. It

compared the number of queries about influenza with traditional flu surveillance systems,

such as the CDC process in the United States. Google tracked the searches for appropriate

terms by geographic area in the United States between 2003 and 2008 and compared it

to publically available data from the CDC's U.S. Influenza Sentinel Provider Surveillance

Network. Google researches found not only that the search results were verified by the CDC

data but also that the search results

predicited

the CDC data. That

is,

because people search

for symptoms prior to seeking a physician's care (from which the CDC data are compiled)

and because it is so much faster to process the search data than the physician's data, Google

could predict the outbreaks

by region

up to two weeks earlier than the CDC. The data are

shown in Figure 4.10.

4

If

the

data continue to provide the same predictive capabilities, they

could predict pandemics or epidemics sooner and thus give health professionals a longer

window to stem the negative effects.

A famous example of data mining is from a chain of midwestern (U.S.) grocery stores

and the purchasing data of their customers. They found male customers generally shopped

on Thursdays and Saturdays. Further they found that these men tended to do their weekly

shopping on Saturdays, but only purchased a few items on Thursdays. Further analysis of

what

they purchased showed that men who purchased diapers on Thursdays also tended to

purchase beer. Armed with this result, the grocery chain made sure the beer display was

close to the diaper display

and

that both diapers and beer were sold at full price on Thursday

to maximize their revenues.

4

More information about Google's work with flu trends can be found at http://www.google.org/

flutrends/. Information about their other data-mining

activities

can

be

found at http://www.google.org/.

DATA MINING

149

Figure 4.10. Google results. Adapted from Google's Flu analysis, available at http://www.

google.org/flutrends.

Of course, not all data mining is good. Consider the research conducted by Acquisti

and Gross (2009) of Carnegie Mellon University. They showed that it is possible to predict

narrow ranges in which a given person's social security number is likely to fall simply by

mining public data. In particular, they used the Social Security Death Master File, which

includes information about those people whose deaths have been reported to the Social

Security Administration. Since this file is a popular tool among genealogy researchers

for finding ancestors, it is readily available online and easy to search. The researchers

mined the data to detect statistical patterns in Social Security Number (SSN) assignment

associated with date of birth (and thus likely date of application for a SSN) and location of

birth. Using their results, they were able to identify the first five digits of

44%

of deceased

individuals born in the United States from 1989 to 2003 and

complete

SSNs with less

than a thousand attempts for almost 10% of those deceased individuals. With that tool,

it becomes statistically likely that they could predict with the same level of accuracy for

living individuals. The professors are interested in the mining algorithms and the public

policy implications; however, in the wrong hands, this could provide the keys needed for

identity theft.

Although data warehouses provide access to information that will help decision makers

understand their operations and environment better, users can become lost in the enormous

possibilities for analysis and miss the forest for the trees. These efforts require the co-

ordinated efforts of various experts, stakeholders, or departments throughout an entire

organization. Available tool users mine the value of the information available in these

warehouses to find the kinds of data that seem to discriminate among alternatives the best,

identify cases which meet some criterion, and then summarize the result or find patterns in

the data to highlight important trends or actionable situations.

The five approaches to data modeling are given in Table 4.2. In each case, the goal is

to find patterns in the data that we might exploit to improve the business. Knowing what

items customers tend to purchase together, or under what conditions emergency rooms will

need assistance, or when products are sufficiently similar to substitute them, will all help

managers run their businesses better. It requires that the system search for patterns in the

data and then differentiate the patterns that are interesting and useful from those that are

illusions and spurious. Said differently, the goal is to find a model that generates predictions

150

MODEL COMPONENT

Table 4.2. Data-Mining Goals

Classifications

Clusters

Regressions

Sequences

Forecasting

that are most similar to the data on which you build the model. At the same time, however,

the focus is not on the training data, but rather on future data. If you overfit to your training

data, then the patterns are likely to perform less well on test set data. Said differently, it

provides a model that is specific to the random fluctuations in the original data. When

applied (which is always the goal), over-fit models tend to do poorly because the new data

experience different "random fluctuations." Hence it is important to have "pure" (not used

in the original analysis) data on which to test any mining model before using it to impact

business rules. Netflix understood that, but apparently some of the contestants did not.

The most commonly used data-mining technique is

classification.

Classification iden-

tifies patterns that are unique for members of a particular group. It examines existing items

that have already been classified and infers a set of rules from them. For example, the

system might classify attributes of those students who complete their degree in a specified

number of years from those who do not. By finding the similarities among students who

do not successfully complete their degrees, the system might find "early warning signals"

on which the administration can act.

Classification mining can produce misleading results, particularly if not done properly.

For example, one of the most controversial classification efforts was the Total Information

Awareness Program (ITAP) of the U.S. Department of Defense. The original goal of the

Nctfiix is known for using quantitative analyses for improving its performance. In 2006 it an-

nounced a

S1

million competition to the first team that could improve it recommendation system

by 10%. The recommendatio

n system, which is used to suggest movies to individual customers,

predicts whether someone will enjoy a movie based on how much tbey liked or disliked other

movies. Netflix provided anonymous rating data for mining and a test data set to evaluate how

closely predicted ratings ot movies match subsequent actual ratings. This set off a flurry of activity

of individuals

, groups, and groups of groups, fn mid-2009

:

a team called BellKor's Pragmatic

Chaos was the first to achieve the goal of improvin

g the system by 10Ό9%

+

According to the

rules,

the other teams had 30 days to improve upon BellKor's method. Just before the deadlin

e

was reached another team, The Ensemble, submitted a method that improved the rating system

by 10,10%, BcllKor did not have time to respond.

However, shortly thereafter

, the team's captain, Yehuda Korcn posted a note on his blog that

he was contacted by Netflix and was told they have the best test accuracy and should be declared

the winner. Why? It appears that Netflix kept two verification test sets: one that was the basis

for the public standings and another that was secret The winner was selected based on the success

of the approach on the

secrer

data set. So BellKor, which appeared to come in second, based

on the public verification test set, seems poised to be the winner based on the hidden test set

Apparently The Ensemble got their additional improvement by overfilling their algorithm to the

test data set; when tested on the unused data, their algorithm was inferior.

DATA MINING

program was to examine large quantities of data, from telephone calls and credit card

purchases to travel and financial data, to detect data that would identify potential terrorists.

TIAP was to use both supervised and unsupervised learning to identify "people of

interest." Supervised learning might find rules linking certain fields in the databases with

known terrorist behavior. Using this method, the mining algorithm might identify all indi-

viduals from certain countries who enrolled in flight school but did not learn how to land

and see what else they had in common. Examination of the additional fields might help

decision makers identify those having terrorist intentions. Unsupervised learning might

find people engaged in suspicious activities that are not necessarily terrorist oriented but

are unusual and should be investigated.

The program was quickly canceled because of the concern about constitutionality of

abuse of the privacy rights of

U.S.

citizens associated with the program. But, if it were not

cancelled, could it work?

This project highlights some of the difficulties of data mining.

False

Positives.

In practice, any time you try to classify people, some will be incorrectly

classified. Some people who should, using this example, be classified as terrorists

would not be (called a false negative). Further, some who should not be classified

would be classified as terrorists; that is a false positive. Even rules that were 99%

accurate (and that level of accuracy would

be

phenomenally unlikely) would identify

a substantial number of false positives. Consider that when looking at 200 million

individuals a

1 %

error rate still generates 2 million false positives. That would result

in not only possible negative impacts on a large number of lives but also a lot of

wasted investigation time.

Insufficient

Training

Sets.

Fortunately, there have only been a small number of instances

of

terrorism.

With such small data sets, the resulting rules would be far less accurate

than the 99% identified in the previous point.

Pattern Changes.

Following this approach, all analyses are done on historical data.

Any behavior changes in the terrorists over time would not be represented.

Anomalies.

People sometimes change their behavior for perfectly good reasons having

nothing to do with terrorism. So, even though they may fit a "profile" for a terrorist

(or for a fraudulent charge), it may have nothing to do with terrorism.

Because the costs of being wrong are so high in this situation and because of the constitu-

tional issues, the program was stopped. But these same issues can impact any data-mining

situation and need to be addressed before decisions are contemplated.

A similar process is

clustering.

The process identifies clusters of observations that

are similar to one another and infers rules about groups that differentiate them from other

groups. It differs from classification, however, in that there are no items a priori classified,

and hence the model needs to determine the groupings as well. A university might cluster

students of similar performance in a class for the purpose of the studying what pastclasses or

experiences they share that might explain their similar performance. Credit card companies

regularly cluster records to determine which customers are likely to respond to different

incentives or even which charges are likely to be fraudulent.

A third kind of data mining is known as

regression.

The goal of this kind of data

mining is to search for relationships among variables and find a model which predicts

those relationships with the least error. For example, a supermarket might gather data of

what each customer buys. Using association rule learning, the supermarket can work out

152 MODEL COMPONENT

Modeling Insights

nderstandin

g single-malt Scotch Whiskey

Single-malt Scotch whiskeys are an acquired taste. They are distilled from barley at a single

distillery and matured in oak casks for at least three years (some for many years). Scotch whiskeys

cannot be matured in new oak casks because the new oak would overpower the taste of the whiskey,

so it is only matured in used casts, Clearly the previous use of the cask will impact the taste of

the Scotch whiskeys. The taste of American bourbon in oak will impact the taste differently than

will Portugese port or by Spanish sherry or Carribean rum or maderia. Similarly, each year that

the Scotch whiskey is in the cask will change the taste since it continues to process. The water

supply will also impact the taste of the final product.

Single-malt Scotch whiskeys tend to be categorized by the region in which they were

produced, While this is useful for those who really know their whiskey, it is less useful for the

general public. So,

a

project called Whisky Classified develope

d

a

clustering system to help people

understand styles of the common brands. Said differently, the project helps someone answer the

question, ^if I Like this brand, what other brands am I likely to like?"

The developers reviewed tasting notes in recently published books on malt whiskey and

from distilleries, From this, they developed a vocabulary of 50U aromatic and taste descrip-

tors for Scotch whiskey. They applied these terms to 86 single-mall Scotch whiskey using

a product called ClustanGraphics. The cluster analysis groups malts into the same cluster

when they have broadly the same taste characteristics across all 12 sensory variables. Tech-

nically, the method minimizes the variance within clusters and maximizes the variance between

clusters.

The result was 10 clusters of single-malt Scotch whiskeys:

Cluster

A:

Full-Bodied

, Medium-Sweet, Pronounced Sherry with Fruity, Spicy, Malty Notes

and Nutty, Smoky Hints

Cluster B: Medium-Bodied, Medium-Sweet, with Nutty, Malty, Floral, Honey and Fruity

Notes

Cluster C; Medium-Bodied, Medium-Sweet, with Fruity, Floral, Honey, Malty Notes and

Spicy Hints

Cluster D: Light, Medium-Sweet, Low or No Peat, with Fruity, Floral, Malty Notes and

Nutty Hints

Cluster E: Light, Medium-Sweet, Low Peat, with Floral, Malty Notes and Fruity, Spicy,

Honey Hints

Cluster F: Medium-Bodied, Medium-Sweet, Low Peat, Malty Notes and Sherry, Honey,

Spicy Hints

Cluster G: Medium-Bodied, Sweet, Low Peat and Floral Notes

Cluster H: Medium-Bodied, Medium-Sweet, with Smoky, Fruity, Spicy Notes and Floral,

Nutty Hints

Cluster I; Medium-Light, Dry, with Smoky, Spicy, Honey Notes and Nutty, Floral Hints

Cluster

J;

Full-Bodied, Dry, Pungent, Peaty and Medicinal, with Spicy, Feinty Notes

Those who want more information about the exercise and especially advice about other Scotch

whiskeys they might enjoy should consult Wishart (20O6).

Adapted from

Wishart,

D.,

Whiskey

Classified*

London,

Pavillion,

2006*

Materials used with

the

permissio

n

of

Mr. Wishart and Pavillion, and imprint of Anova Books.

DATA MINING

what products are frequently bought together, which is useful for marketing purposes. This

is sometimes referred to as "market basket analysis." One uses this kind of mining to

find associations among the factors. Associations are events linked with regard to a single

criterion, such as two or more courses that students tend to take together, such as DSS and

database systems. The fact that students take the courses together might not be apparent

without the analysis. However, after the analysis, we know that the two courses should not

be scheduled at the same time.

Sequences

are events linked over some period of time, such as patterns the students

employ for taking courses over multiple semesters. The important characteristic of these

linkages is that they are ordered: observations with characteristic X are also likely to have

characteristic

Y.

For

example,

a student who takes a statistics course this semester

is

unlikely

to take the forecasting course for two subsequent semesters. This will help the department

plan course offerings. Or, perhaps more commonly, voters who express interest in issues

of education and health care prior to the election are more likely to vote for the Democratic

candidate.

Finally,

forecasting or predictive

data mining is the process of estimating the future

value of some variable. While clearly it is possible to use tools like regression to forecast

the future, the goal of the data mining is to find rules that might predict what will happen.

Universities do (or should do) this kind of mining since they have significant historical

databases of students, their characteristics prior to admission, and their level of success.

So a data-mining exercise might identify specific combinations of test scores, experience,

and grades that were associated with successful students (generally defined as those who

graduate) to find decision rules for admissions. Insurance companies mine their data of

symptoms, illnesses, and treatment plans and outcomes to determine the best course of

treatment for particular illnesses. In the latter case, this analysis might be with regard to

outcome and to cost.

An interesting form of predictive data mining is in the area of text mining. This can

be particularly useful for brainstorming or alternative generation in the decision-making

process. Suppose, for example, that you are the state senator on a transportation committee

and you are trying to determine what projects are most important to your constituents. Of

course you can read everything on the Internet about it or you can poll your constituents, but

both of those take time. Instead you want to have the computer analyze some transportation

blogs on the subject of transportation in your state. One way to analyze the blog is to input

the text of the blogs in a

a

product such as IBM's "Many Eyes" so it can analyze the words in

the

text.

A starting point might

be

to examine

a

word cloud such

as

that shown in Figure

4.11.

The word cloud sizes the words in proportion to the number of times that they appeared

in the blog. You can see from this that the bloggers discuss specific locations, such as St.

Louis or Jefferson City, and individuals in the MoDOT hierarchy most, because those words

are the largest. Moving beyond this, you see that terms such as "bridges," "safety belts,"

and "work zones" appear frequently. To pursue those lines further, consider a word tree that

gives more information of how those words are used in context. An example that shows

phrases following the word "bridge" is shown in Figure 4.12. Many Eyes will allow users

to click on the various terms and follow them to their completion or do additional analyses

on them. The goal of the use is, of course, to provide ideas to the senator about what is

important to his or her constituency.

Data mining can be a very useful tool for identifying trends that decision makers might

not have considered. However, it can also identify statistically significant trends that are not

the least bit useful. The decision maker needs to understand the assumptions underlying the

statistics and the implications for their data before applying the results from data mining.

Figure 4.11. Word cloud analysis of a blog discussing plans and problems of projects under consideration by Missouri DoT Department of Transportation.

The summary was prepared by David Doom using IBM's

tool,

"Many Eyes."

Figure 4.12. Word tree of a blog. Word bridge analysis of a blog discussing plans and problems under consideratio

n by Missouri DoT. The summary was

prepared by David Doom using IBM's

tool,

"Many Eyes."

156

MODEL COMPONENT

For example, large sample sizes can result in even very small differences to be statistically

significant. Even if you know that a rare event is statistically more likely under certain

circumstances, it might not change how you approach a decision. If it does not change the

decision, it is not important. Also, statistical significance does not address the question of

the cost of gaining and using the intelligence. If the cost of applying the rule is greater than

the savings associated with ignoring it, even if it is statistically significant, the exercise

does not imply that decisions should be changed. Finally, there is the problem of running

many tests. If you test enough hypotheses, about 5% of them should be "significant" even if

they are all false. That is what the significance level means. So, if the tests are random, and

not based upon some reasonable understanding of the business, some results might simply

reflect spurious relationships that are not useful for running the business.

A variety of analytical tools—neural networks, decision trees, rule induction, and data

visualization—as well as conventional analyses are used to complete these

five

kinds of data

mining. These tools can "learn" to predict changes in the environment, generate rules for

classification of

data,

find similar subjects among the data, identify if-then rules for action,

and display data so that decision makers can glean important

patterns.

To be successful, the

approach and the product must meet the needs of the user and a particular data warehouse.

Other criteria for the evaluation of data-mining products are listed in Table 4.2.

Intelligent Agents

Intelligent agents are pieces of software that complete specific, repetitive tasks on behalf

of

the user. They are not new to computers; in fact, they are commonly in use on systems to

monitor CPU and peripheral use and capacity. Other intelligent agents are associated with

e-mail

systems, where they help sort and prioritize

e-mail

by sender or topic on behalf of

the user. Their new use is as a means to search through relational databases to find relevant

data for decision makers. Even more exciting is the combination of search protocols with

analytical capabilities that will cause the intelligent agent not only to find data but also to

analyze it to find examples of trends or patterns the decision maker might miss on his or

her own. In addition, the intelligent agent can get at the information faster to detect unusual

occurrences so the decision maker can act upon them more quickly.

For example, consider the product DSS Agent from MicroStrategy, as shown in Figure

4.13.

This product surfs a data warehouse for information and summarizes it for decision

makers. In fact, this particular screen summarizes data at Recovery.gov, which in 2009 con-

taied information about how the American Recovery and Reinvestment Act was working,

including an up-to-date data on the expenditure of funds.

Researchers at the University of Vermont developed a website, http://www.weteelfine.org4

s

that

mines through some 2.3 million blogs looking for sentences beginning with

tl

I feel" or "I am

feeling." They use personal online writing to determine the mood of people in

real

rime.

After

mining the sentence, they use the standardized "psychologica

l valence" of words (established by

the Affective Norms for English Words) to give each sentence a happiness score, The rating of

the individual blog is not important; rather their goal is lo measure the big picture of a town or

other grouping of

people.

They use their tool in an exploratory fashion to measure the feelings of

the country as a whole. Clearly such a tool could be used to mine for other words, such as those

of a company's product, to provide decision makers with consumer's attitudes about the product,

DATA MINING 157

Satoshi Kanazawa, a reader in management and research methodology at the London School of

Economics, published a series of papers that predict the sex of one's baby, the last of which

is "Beautiful Parents Have More Daughters"*, Dr, Kanazawa took a sample of almost 3000

individuals who were asked the number of children of each gender and who were rated on a

five-point scale regarding attractiveness. His results are shown in the following graph as the

points.

60%

-, 1

Two researchers reexamined his method and found that the "statistical significance

" noted

in the original paper just did not exist.f

Note that the least attractive people (rated 1) had about a 50-50 chance of having a girl

while the most attractive people (rated 5) had about a 56% chance of having a girl. What the

author did was to compare the aggregate of groups 1-4 to group 5 and found that the difference

between them was significant. But, in reality, a correct statistical test would have made not only

that comparison but also other combinations of groups, such as group I to the aggregate of groups

2-5,

or the aggregate of groups

1

and 2 to the aggregate of groups 3-5, and so on. Furthermore, if

you do those additional tests, they

must

be included in the test of significanc

e of the experiment,

In other words, statistical validity relies not just upon the one comparison but rather on

all

of

the comparisons

together.

As the authors point out, the curved lines in the diagram above are the

result of a better test; this test does

not

show statistical significance. This is one of the examples

of statistical problems associated with the mining of data.

*From S. Kanazawa, "Beautiful Parents Have More Daughters: A Further Implication of the Generalize

d

Trivers-Willard

Hypothesis,'*

Journal

of

Theoretical

Biology,

244,

2007,

pp.

133-140.

r

From A. Gelman and D. Weakllem, Of Beauty, Sex and Power,"

American

Scientist,

97(4), July-August

2009,

pp.

310-314,

I

158

MODEL COMPONENT

Figure 4.13. DSSAgent screen. Summary of the progress of the American Recovery and Reinvstment Act available

at: http://www.microstrategy.com/recovery-act-data

. Image is used with permission from microstrategy.

Using these intelligent agents, users can schedule intelligent agents to execute on

a one-time basis, periodically, or based upon events. For example, decision makers can

perform regular scanning of absenteeism or missing reports to highlight indicators that

problems might need attention. Or, decision makers can schedule intelligent agents to

find information about changes in demand after planned promotions or after a particular

indicator reaches some prespecified value. Using workflow triggers, users can specify both

pre-

and postagent macros that can integrate with other modeling components of the DSS.

For example, the agent could find information that would automatically be imported to a

forecasting application to compute projected demand. If desired, another agent could be

triggered to mail results of the application automatically to people on the management team.

Many intelligent agents today provide a set of options through which the user can scan

the data warehouse. For example, users can define filters based upon specific qualifying

criteria. Or, users can define percentile and rank filtering; using this option, decision makers

could identify the source of the top 10% of their raw materials, for example. Similarly,

intelligent agents can be launched using conditional metrics. Hence, users can specify

information to be found regarding a particular business unit and compared it to that of

multiple business units or to the company as a whole.

To fully exploit the data-mining capability, however, the intelligent agents need to be

combined with artificial intelligence so the software can find not only the data but also

MODEL-BASED MANAGEMENTSYSTEMS

159

the patterns in the data. In fact, if it works well, data mining should find answers in the

data which the decision maker has yet to consider asking. Data-mining tools find patterns

in the data, infer rules from them, and then refine those rules based upon the examination

of additional data. The patterns and rules might provide guidelines for decision making

or they might identify the issues upon which the decision maker should focus during the

choice process.

MODEL-BASED MANAGEMENTSYSTEMS

The DSS provides the decision maker with more than the models themselves. Through

the Model Base Management System (MBMS), the DSS provides easy access to models

and help in using those models. Clearly, the library of models is an important aspect of

this component. Such a library should provide decision makers access to a wide variety

of statistical, financial, and management science models as well as any other models that

could be of importance to the particular problems to be encountered.

Easy Access to Models

The library of models is provided so as to allow decision makers

easy

access to the models.

Easy access to the models means that users need not know the specifics of how the model

runs or the specific format rules for commanding the model. For example, consider the

screen from the SAS Data Miner module, shown in Figure 4.14. In Figure 4.14, we can see

that users can easily select a model simply by clicking on a tab shown at the top. In Figure

4.15,

which shows an application of IBM's Cognos, we see how the user can manipulate

the tools once they are chosen with simple keystrokes or mouse movements.

The MBMS should facilitate easy entry of information to the model. Unlike conven-

tional modeling software, which often requires that information be entered in a specific

order and a specific format, DSS should allow flexible input of the data. The role of the

MBMS is to translate the user-friendly form of the data into the appropriate format for a

Sentiment analysis is the effort to translate human emotion into data that can be used by decision

makers lo understand their

clients.

It

is,

in essence, the data mining of blogs and social networks to

examine and summarize reviews, ratings, recommendations* and other forms of personal opinion.

The tools attempt to categorize statements that are straightforward, such as"[ love this product" or

"I hate this movie," as well as those using sarcasm, irony, and idioms. Filtering through hundreds

of thousands of websites, these algorithms identify trends in opinions and some even identify

influential opinion leaders. Such tools could help companies pinpoint the effect of specific issues

on customer perceptions, helping them respond with appropriate marketing and public relations

strategies. For example, when Lhere was sudden negative blog sentiment against the Yankees, they

turned to sentiment analysis to identify the issue. The sentiment analysis identified a problem

associated with

a

rain-de laved Yankees-Red Sox game. Stadium officials mistakenly told hundreds

of fans that the game had been canceled, but their electronic ticket vendor denied fans' requests

for refunds on the grounds that the game had actually been played. Once the issue had been

identified, the company offered discounts and credits to the affected fans

and

reevaJuated its bad

weather policy.

160

MODEL COMPONENT

with permission of SAS Institute Inc., Cary NC, USA. (Source: http://www.sas.com/presscenter/screenshots.html.)

particular model. For example, even if a model requires the data be input in a rigid line and

column framework, such as shown below,

1.22 15 3

2.31 21 6

3.11 11 9

the user can input them (if they are not already in a database) flexibly in a format that might

be more comfortable, such as 1.22, 2.31, 3.11, 15, 21, 11, 3, 6, 9. The MBMS will put the

data in the format appropriate for the particular model(s) being used.

Similarly, users of the system need not be aware of the specific syntax required to

execute a particular model. The MBMS should generate the necessary commands to tell

the machine where the model is located and what commands are necessary to cause the

model to execute. For example, the user should not need to remember (or even know) the

requirements for naming or formatting the data to utilize them in a model. Rather than

the user needing to remember the code, such as that shown in Code 4.1, the user would

simply "click" on the icon for accounts data. Clearly, someone would need to program the

system to associate a particular icon with a given place in the database. More important

MODEL-BASED MANAGEMENTSYSTEMS

161

Figure 4.15.

Simple manipulation of a model. Screen shot from the 4.02 mark of the Cognos

video,

"Forecasting in Turbulent Times":http://www-01.ibm.com/software/data/cognos/solutions/

software-reporting-analysis.html; Image is reproduced courtesy of International Business Ma-

chine Corporation http://download.boulder.ibm.com/ibmdl/pub/software/data/sw-library/cognos/

demos/od.forecasting/rollingforecasts.html

.

from the perspective of the MBMS, though, is the fact that the data have been identified in

the appropriate format as input to a particular package (in this case, SAS).

Further, it is important that the program be notified that there is something "unusual"

about the data, such as the record length. Not only might users be unaware of the appro-

priate syntax through which to share this information, they might not even know that the

information needs to be provided. Similarly, users should not need to remember the control

sequences for testing hypotheses (Code 4.2); they could simply type

is there a difference

in absenteeism in the different groups?

Of course, in order to provide this easy access to models, the designer must make

certain assumptions about how the decision makers want their analyses conducted. In this

case,

the designer made assumptions about the specific test of the differences of means

among the groups by specifying the model, the test, the procedure, and the format of

output. On the one hand, this makes analysis easier for the decision makers because they

can access the model immediately without needing to specify assumptions, look up syntax,

Code 4.1 Sample Code to Input Data from a Modeling Package

CMS FILEDEF ACCOUNTS DISK ACCOUNT DATA AI  (LRECL 135); 

DATA SAMPLE; 

INFILE ACCOUNTS; 

INPUT DEPARTMENT $ 1-7 EMPLOYEE $ 9-2

5

 NUMBER 27-32 

ABSENT_FUL

L 34-36 ABSENT_HALF 38-42 REASON 80-133; 

TOT_ABSENT =  ABSENT_FULL+ABSENT_HALF

;

162

MODEL COMPONENT

or write code. On the other hand, it constrains those decision makers who need different

assumptions for their particular test. This presents somewhat of a dilemma for the designer

of the system in knowing how to make the trade-off between flexibility and control.

Regrettably, there is not a standard answer to this question, and only knowledge of the

decision makers, their preferences, their agreement on their preferences, and the likelihood

of their changing preferences will define how much flexibility is needed in the model

features. However, a designer can compromise. If, for example, most decision makers want

the features set in a particular way but not all accept this option, the features could be set

with a default setting and easy access to change the settings. Upon the selection of the test,

a window such as that shown in Figure 4.16 could appear. As the users click a mouse (or

press enter) on any one of those, they would see another window that allows them to change

the options.

There are variations on this approach. If, for example, the differences in features

is person specific, the designer could build intelligence into the system with a rule that

specifies that, if the user is PERSON X, the Gabriel test rather than the Duncan test should

be used. In this way, PERSON X always has the preferred test as the default and all others

have their preferred test as the default. Or, the designer could provide a check box that

Figure 4.16. Model option selection.

Code 4,2 Sample Code to Process Data from a Modeling Package

PROC ANOVA; 

CLASS 0FFICE1 OFFICE2 OFFICE3; 

MEANS 0FFICE1 OFFICE2 OFFICE3 OFFICEl*OFFICE3/DUNCA

N LINES; 

MODEL Y =  OFFICEl  | OFFICEl*OFFICE3  | 0FFICE3/INT INTERCEPT; 

TEST H = TOT^ABSEN

T TOTJ^SENT*SENQRITY  JOB  

TITLE  'ABSENTEEISM BY OFFICE, SENIORITY, JOB'; 

CLASSES

MEANS

MEANS TEST

MODEL

INTERCEPT

HYPOTHESIS TESTS

Hr

E:

defined as A B C

specified as A B C andA'C

Duncan

defined as main effects

tested as an effect

will be printed

specifies numerator effects

specifies denominator effects

MODEL-BASED MANAGEMENTSYSTEMS

would allow users to change defaults before running the test if they desire. While it is

tempting to force the user to acknowledge and accept each option individually, it is not

recommended. Such a sequence will increase the average amount of time it takes for a

user to run a model. Unless many users often change the options, this is an unnecessary

waste of time. In addition, many users will quickly tire of these repeated entries, learn to

ignore them (by pressing accept for each option), and become frustrated with the system.

Furthermore, they will not be any more likely to actually read the entries.

Understandability of Results

In addition, the DSS should provide the results back to the user in an understandable form.

Most models provide information to the user employing at least some cryptic form that is

not comprehensible for people who do not use the package frequently. For example, the

results from a regression could be presented using a standard output format of a commonly

used modeling package, such as that shown in Figure 4.17. Regular users of this modeling

package can find most of the information that they need to evaluate the model and begin

forecasting with it. However, even a person who is familiar with statistics but unfamiliar

with the output of this package or other statistical packages might not be able to interpret

the meaningfulness of the results. Certainly a decision maker not familiar with either

statistics

or

the modeling package would be unlikely to be able to answer even the simple

question of how many items one would expect to sell if the price were $1.24 and the

advertising expenditures were $15/month. Consider, instead, a screen such as that shown in

Figure 4.18.

In Figure

4.18,

the results are labeled clearly and all the relevant information

is

provided

to the user in a conclusion format. The user does not need to remember too much about

the technique "regression" because the screen explains the types of issues that should be

of interest. Furthermore, it encourages the user to experiment with the model (by entering

data) so as to become more comfortable with it and the

results.

Since one of the fundamental

assumptions in DSS design is that the supported decisions are "fuzzy" and infrequently

encountered, it is important not to assume that the user can remember the nuances of the

output of each model that might be accessed.

Note that we are not simply talking about the

appearance

of

the

results.

In Figure 4.18,

we are literally helping the user to understand the

meaning

of

the

output by removing some

of the jargon implicit in the computer printout and rephrasing in terms the decision maker

can understand. For example, consider the boxed information on the left. The purpose

of the box is to highlight the meaning of the slope coefficient associated with each of

the variables as well as their associated interval estimates. In contrast, Figure 4.17 lists

the slope in the column "parameter estimate" next to the respective variable name. The

appropriate standard error appears in the following column. To use the information from

the the modeling package output, the decision maker needs to know what each of these

terms means and that a slope can have a physical interpretation. Furthermore, the decision

maker needs to know that all point estimates have intervals associated with them and that we

determine the interval by multiplying the standard error by the critical value of

t

associated

with 48 degrees of freedom, which is found in a standard

t

table but not in Figure 4.17!

This is a lot to expect from the decision maker, especially given that each model has its

own unique notation and set of

issues.

The box in Figure 4.18 does not require the decision

maker to know all the intermediary steps or to compute anything. In short, Figure 4.17

provides results from the model. Figure 4.18 provides

support

for a decision.

Clearly, different individuals will require different levels of support. Figure 4.18 pro-

vides only the minimal quantitative information. However, it can be tied to other output

164

MODEL COMPONENT

Figure 4.17. Traditional results format.

screens that could provide additional support if the decision maker selects it. For exam-

ple,

in Figure 4.21, the instructions note that the user can obtain additional information

about a specific topic by clicking the mouse on that statement. In this screen, the statement

"both variables are useful" is highlighted. If the decision maker clicked on that space, the

system would display Figure 4.19, which provides additional information, including the

mathematics and assumptions behind the statement.

The previous example provides information to the decision makers only if they select

it. However, sometimes you want to make sure that the decision maker sees additional

help screens because it is crucial. In this case, the system can "force" a particular area of

the screen to be highlighted, create a "pop-up" notice about a problem, or emit a sound

to catch the decision maker's attention. Suppose, for example, the variable "price" in the

model described in Figure 4.20 were

not

statistically significant. It is possible to provide

the information in a box as shown in Figure 4.20.

This box provides information about the validity of the model. However, it is passive

and does not highlight the problem or tell the decision maker the implications of the

problem. Instead, consider Figure 4.21. In this screen, we are highlighting some of the

information so that it is not missed by the decision maker. Not only does this additional

screen call attention to the easily missed note about the variable being not statistically

significant, the "CAUTION" screen tells the decision maker the implications of not taking

MODEL-BASED MANAGEMENT SYSTEMS

165

Figure 4.18. Results with decision support.

action on this problem. In this way, the DSS is helping the clients clarify their assumptions

about the implications of the results. So, in fact, the DSS is helping the decision maker to

use the information correctly.

The way we accomplish this task depends on what kind of DSS generator and modeling

package we are using. In an abstract sense, there must be code that causes the computer to

scan the results of the model and creates the base screen with the results. In this case, the

modeling package must return the results of the

F

statistic, the

t

statistics, the probabilities

associated with those

t

statistics, and the mean squared error. Further, there must be some

"intelligent" link that fires to interpret the results and to place those results in the appropriate

window. Finally, there must be another intelligent link that fires when one of the variables

is not significant to cause the "CAUTION" screen to appear.

Clearly, creating this kind of help in a traditional language is difficult. The fourth-

generation languages and object-oriented languages available today allow the designer

much more flexibility. First, such languages allow the user to create "pop-up" windows

that are linked to particular results or variables. In this case, each of the four items noted

in the results window might actually be a different window that is linked to code checking

the appropriate result. The border might actually be a hyperregion that serves no purpose

but an aesthetic one. Furthermore, the "CAUTION" screen might be linked to an indicator

of nonsignificance of a variable. An alternative "CAUTION" screen might be linked to a

condition where two or more of the variables are not significant.

166

MODEL COMPONENT

Figure 4.19. Detailed model support.

Integrating Models

Another task of the MBMS is to help integrate one model with another. For example,

suppose the user needs to make choices about inventory policy and selects an economic

order quantity (EOQ) model, as shown in Computation

4.1.

To

use this formula

to

determine

the optimal order quantity, we need information about expected product demand, the costs

associated with an order, and the typical holding costs (with consistent monetary and time

units).

If the decision makers can input the data or read the data directly, there is no problem.

Typically, however, this is not the case. Generally, the order costs need to be computed by

combining the costs of personnel, supplies (such as forms), and services (such as phone

resources) needed to execute an order. In addition, since holding costs can vary over time,

we need to average holding costs to obtain a current estimate. Finally, unless demand is

MODEL-BASED MANAGEMENTSYSTEMS

167

Figure 4.20. Passive warning of model problems.

well specified, it needs to be forecasted based on historical data. Hence, upon selection of

the EOQ model, the MBMS needs to complete several tasks:

1.

Search the database for a single value for the order costs.

2.

If

no

specific order cost information is available, invoke the model to compute order

costs by summing personnel costs, supply costs, service costs, and the order cost

charged by the vendor.

3.

Feed the computed order costs to the EOQ model.

4.

Obtain data about holding costs.

5.

If historical data are available, estimate holding costs.

6. If no historical data are available, invoke the model to determine holding costs.

7.

Feed the computed holding cost value to the EOQ model.

8. Invoke the model to forecast demand for the time period(s) served by the order.

9. Feed forecasted demand to the EOQ model.

10.

Compute the economic order quantity.

The user not only should not need to intervene in this process but also should not

need to know the process is occurring. However, since the meaningfulness of the EOQ is

dependent upon the quality of the forecasts and estimates, the user should be provided the

168

MODEL COMPONENT

Figure 4.21. Active warning of model problems.

forecasts and information about the quality of those forecasts. This might be accomplished

as in Figure 4.22.

Sensitivity of a Decision

One of the tasks of the model base management system in a DSS is to help the decision

maker understand the implications of using a model. This is not always easy because

decision makers may not be inclined to ask questions, particularly if they do not know what

questions need to be asked. Consider the following examples.

Example 4.1. Peara's Personalized Widgets uses an assembly line to build desired con-

figurations. One of the employees on the line has suggested a change in procedure that

Andrew Peara thinks might improve the efficiency of the operations. Andrew Peara wants

to determine if his intuition is correct and if the change would be worth implementing. To

investigate this using historical data, he determines that the mean length of time to perform

a certain task or a group of tasks on an assembly line is 15.5 minutes, with a standard de-

viation of

3

minutes. Because he understands the importance of collecting data, he selects

16 employees and teaches them the new procedure. After a training period, he finds these

employees, on average, take 13.5 minutes to perform the task with the new procedure.

The question Andrew needs to answer is whether these results provide sufficient evidence

MODEL-BASED MANAGEMENTSYSTEMS

169

Figure 4.22. Integration of models.

to indicate that the new procedure is really faster and thus should be implemented. This

statistical analysis for this problem is shown in Computation 4.2.

Computation 4.2. Sample f-Test

In introductory statistics, you learned that this type of problem is a one-tailed test of the mean.

From a statistical point of

view,

the question is

Ho:

H

A

:

μ=

15.5

μ < 15.5

Where μ is the true mean task time. To test this, given the sample size of 9 and the estimated

standard deviation, one uses a

t

test: Reject

HQ

if computed

t

is

less

than the critical

t

value,

ti£ =

1.8331, or

t

=

μ

tf^fn

<

1,8331

In this problem,

t

=

■

ß

13.5-15.5

= -2

Since the calculated value for /is less than the critical value of

t

(in standard r tables found as

—

1,8331

)>

one can reject the null hypothesis,

MODEL COMPONENT

Based on the analysis, Andrew Peara knows that there is reason to believe the new

procedure will reduce the amount of time it takes to perform the task. However, it is unlikely

that this is the only information the decision maker will want to know in order to make the

decision. It is obviously necessary to determine whether the value of

the

additional widgets

that could be produced (because it takes less time to perform each one) offsets the cost

associated with the training. We could then estimate that instead of producing 3.87 widgets

per hour (one every 15.5 minutes), the average person will be able to produce 4.44 widgets

per hour. Said differently, this is an increase of 4.59 widgets per shift, or 22.98 widgets per

week for an average worker. With this information and some information about the revenue

per computer and the cost of training, the decision maker can easily decide whether the

additional 23 widgets per week per worker will increase revenue sufficiently to justify the

costs of training.

However, this analysis is built upon some assumptions that may not be clear to the

decision maker. One of the characteristics of good decision support is that it helps the deci-

sion maker understand these assumptions and evaluate whether or not they are reasonable.

Before discussing how to display the information, we need to know the assumptions.

1.

A major assumption underlying this analysis is that these 16 individuals really do

represent the employees who will perform this

task.

While the description of the

problem indicated that the 16 were "randomly selected," it is important to be sure

they are representative. In real-world cases, "randomly selected" might mean the

16 people who volunteered, the 16 best workers, the 16 biggest problems for the

supervisor, or the 16 people who happened to make it to work on a very snowy

day. Since you are not provided with information regarding how the sample was

selected, it is important to test whether these employees really were representative

by comparing their task times prior to the introduction of the new procedures to

their times afterward (such as through a paired

t

test).

Consider the three possibilities and how they could affect the decision. If their

average pretraining assembly time were not statistically different from that of the

entire group, then the original conclusion appears valid. If instead their average

pretraining task time were statistically larger than that of the group, the results

are potentially more impressive. This fact should be brought to the attention of

the decision maker as even more evidence that the training is good. However, if

their average pretraining assembly time already were statistically lower than 15.5

(especially if it is statistically lower than 13.5), Andrew Peara would need to know

the training might not be as effective as the test first indicated.

2.

A second assumption is that the variance associated with task completion will

not be increased.

The original description of the case indicated that the standard

deviation is 3 minutes. Since one of the major causes of bottlenecks on assembly

lines is increases in variation of assembly time, it is necessary to determine whether

the posttraining standard deviation is still 3 minutes. Problems in balancing the

line and/or quality control will almost certainly occur with an increase in the

variance.

3.

One of

the

basic assumptions is that there is demand for the extra capacity.

The

benefits of achieving this new efficiency can only be realized if either there is

demand for additional items or the workers can be used profitably in some other

task. If

not,

regardless of

the

results of the test, incurring the cost of the new training

is not worthwhile.

MODEL-BASED MANAGEMENTSYSTEMS

As with most aspects of decision support, there is no universally correct way to provide

this information to the decision maker. The basic options are (a) check the assumptions

automatically and note the results on the screen in a pop-up box; (b) check the assumptions

automatically and only note the violations of the assumptions on the screen; (c) note the

assumptions on the screen and allow users to decide whether or not they need to be checked

(either individually or as a group); or (d) ignore the assumptions and assume the users

know enough to check them without system help. Clearly each option has advantages and

disadvantages. If we provide total information (the results of the tests on the screen), then

the user is informed about the reasonableness of the use of the statistic. However, users

may find this information clutters the screen, especially if many assumptions are evaluated

for a given test. In addition, users may not take the time to scan the information box and

hence may not notice the violations. Similarly, if we simply give the users the option of

checking assumptions, they may not take the time because they do not know the value of

the additional information. However, if the users are quite knowledgeable about their data,

this option saves processing time and hence provides a faster response to the user. By not

warning the users of the potential problems, we fail to provide decision

support.

The remaining option, check the assumptions and list only those that are not validated

by the check, provides the support necessary to help users apply the techniques better. In

addition, since only problems are noted on the screen, the results do not become tedious and

users know they should pay attention to them. Of course, testing the assumptions can use

more processing time and hence slow response time. If this is perceived to be a problem,

we can always allow the user to set options to ignore the testing of

one

or more assumptions

prior to running the test. We even can build these preferences into a profile for each user so

they do not need to be set each time a model is invoked.

In addition to testing assumptions to verify that a model is being appropriately used,

the decision maker might simply want to develop a better intuition for the problem. The

MBMS should help users to investigate more facets of a problem easily. Typically, such

additional analyses are menu options, not automatic procedures.

Consider the types of additional analyses that might be undertaken in the problem of

the mean task times just considered. Clearly, additional analyses are more crucial if the

results of the analysis suggest there is no difference in the two means. Such intuition can

be facilitated by the system giving information about the sensitivity of the results to the

various conditions of the problem. For example, it might be quite reasonable to provide

some information about what mean time would be necessary to produce a statistically

significant result. This can be determined by using the same equation but solving for the

sample mean necessary to achieve the critical value of

t

(from a statistical table), as shown

in Computation 4.3. So, as long as the new procedure takes, on average, less than 13.67

minutes, it will produce a statistically significant improvement. Alternatively, we might

want to know how large a sample would have been necessary to obtain significance with

the result of an average time of 13.5. Again, it is simply an issue of considering the base

formula in a slightly different manner as shown in Computation

4.4.

In this case, the results

172

MODEL COMPONENT

suggest that it was only necessary to have three subjects with the data that are available.

If the test had not been significant and Andrew Peara would want to rerun the test with a

different number of subjects, this equation would tell him how many subjects to select.

Example 4.2.

Consider a second example where a decision maker selects regression to

help solve a problem. In this case, a manufacturer wants to know the relationship between

the age of machinery and the annual maintenance costs. A sample of 50 machines is taken

and the following costs are obtained:

Age

(months) 

Ϊ

Maintenance 

Costs 

Age

(months) 

Maintenance 

Costs 

Age

(months) 

Maintenance 

Costs 

If we constructed a screen for the results of this regression that paralleled that in Figure

4.21,

it would appear as that shown in Figure 4.23. It appears from the information provided in

Figure 4.23 that the model is good and should be used. However, this is not true. Although

the relevant statistical measures of the model have been checked and are significant, they

do not convey the complete story about the implications of using this model. Consider the

graph of the maintenance data shown in Figure 4.24. With a quick examination of the data,

it becomes obvious that there is some phenomenon occurring in the middle of the data.

This change in process is undoubtedly affecting the equation. More importantly, from a

prediction point of view, of course, is the fact that the equation is not particularly good

MODEL-BASED MANAGEMENT SYSTEMS

173

Figure 4.23. Modeling results with some interpretative support.

Figure 4.24. Plot of maintenance data.

174

MODEL COMPONENT

Figure 4.25. Model results with better interpretative support.

at predicting costs for those 10 machines. This suggests that the age of the machinery is

not sufficient to determine maintenance costs and that some other phenomena need to be

considered. From the user's perspective, the graph suggests that while age might be a good

indicator in general, it is necessary to understand the maintenance issue better.

It is difficult, even with today's technology, to have the computer scan the graph and

alert the decision maker to problems in the data. Since the graph conveys information not

communicated by the statistics, it is useful to provide a way for decision makers to get

to the graph easily. If the decision makers can be relied on to look at the information,

simply providing the ability to view the graph through a click of a button is sufficient. An

alternative is to have the graph be part of the screen, as shown in Figure 4.25.

Model Management Support Tools

The kinds of issues associated with model-generated questions like those in the two exam-

ples will, of course, depend upon what model is being used. For example, if the decision

maker is using linear programming to determine a mix of products to produce with a lim-

ited set of inputs, then sensitivity analyses will include questions such as: (a) what if the

company has more of a particular input than specified; (b) what if the company has less of

a particular input than specified; (c) what is the impact on production policies if the price

MODEL-BASED MANAGEMENT SYSTEMS

175

Figure 4.26. Passive prompting for further analysis.

of an input changes; (d) what is the impact on production policies if the selling price is

changed; and (e) what is the impact if we change the relative input needs of the possible

products? Alternatively, if we are using a financial analysis, the questions might be, "How

is present value affected by discount rate, tax rates, or depreciation.?"

Further analyses also might be prompted by a particular result of an analysis. For

example, suppose that the DSS has been created

to

support marketing research for

a

clothing

manufacturer. Suppose further that someone found a result that the demand for the high-end

trousers was declining in some states but increasing in other states. This might prompt the

decision maker to ask questions, such as what do the states where sales are increasing have

in common and what do the states where sales are decreasing have in common. In particular,

the decision maker might be interested in the demographic distribution of the states, the

distribution of competitors in the states, and the similarities in income, population, industry,

or metropolitan areas in the states. Hence, for the system to be effective, the decision maker

should be able to query it about each of these facts. Suppose that in these queries the

decision maker finds the average age of white collar workers is higher in the states where

the trousers are selling well than the states where the trousers are selling poorly. This

provides the decision maker with some information. Perhaps the company officials already

know that their product appeals to more mature clientele. Then, the results probably will

not be investigated. However, if decision makers perceive the product appeals more to

younger clientele, then this information would suggest a need for further modeling to test

the underlying assumptions of their market research efforts.

176

MODEL COMPONENT

Figure 4.27. Active prompting for further analyses.

Perhaps, upon receiving the information regarding declining sales, the decision maker

who is new has no theories about what could be happening. A good DSS should be able

to help those decision makers work through the analyses. For example, it should be able

to prompt the decision maker to consider issues such as demographic changes in the area,

employment trends, costs of living, and other factors specific to that particular product.

Such help might come in terms of a simple "why" key available on the screen, as shown

in Figure 4.27. Or, it might allow appropriate information boxes to appear, such as shown

in Figure 4.28. Alternatively, the decision maker might want to know how the trends are

expected to change over the next five years. Another screen might provide information

about expected trends.

The important aspect of this kind of support is to provide enough of the appropriate

information for the decision maker to understand the phenomenon of interest. The "WHY?"

key might provide information about automatic analyses among predefined options and

display them on the screen. In this way, the decision maker could click a mouse on a

particular statement and identify the appropriate analyses that generated it. The result of

this action might be the display of all related analyses or it might simply be the display

of all significant related analyses. Although each option is appropriate in some cases, a

general rule for selecting between these options is: The higher in management or the less

statistically trained the person, the less nonsignificant analytical results the DSS should

show.

CAR EXAMPLE

177

Figure 4.28. Assistance for defining criteria.

Or the "HELP!" key might provide information about the kinds of analyses that might

be accomplished to further investigate the topic. This differs from the "WHY?" option in

that it allows the decision maker to explore the relationships through whatever analyses are

deemed appropriate. With the "WHY?" option, the user is provided "canned" analyses to

consider. Alternatively, with this option, the system recommends analyses but allows the

user to select either one of the recommended or user-defined analyses. Such an option can

allow an unknowledgeable decision maker to learn more about the decision environment.

It can also allow the very knowledgeable decision maker to pursue some subtle clue that is

suggested by some earlier result.

CAR EXAMPLE

A careful consideration of models for the DSS could result in a system that allows users to

make truly informed decisions. Models should provide support for all phases of decision

making, from the initial generation of alternatives to the final questions of how to finance.

In addition, the model management component should include assistance in the appropriate

use of models and intelligence that reviews model use for errors. Finally, where possible,

the model management system should implement heuristics to automate choices where

decision makers cannot or do not implement choices.

Brainstorming and Alternative Generation

One important model management operation is to help users generate alternatives. At the

simplest level, alternative generation could include searching for options that meet some

MODEL COMPONENT

criterion specified by the user. Some users will want a car that looks "cool" and goes fast.

Others will want a car that will facilitate their car-pooling activities or that will be good for

trips.

Still others will want to consider fuel efficiency or safety in their analysis. Others will

just want a car they can afford. The search process is straightforward and was illustrated in

the previous chapter.

More likely scenarios, however, are that the user is not sure about the criteria he or she

wants to employ or that the user has a general idea of the criteria but does not understand

the specific factors to employ. The DSS should allow users to select any criterion or set of

criteria. However, if we put all possible criteria on a screen, users will find the interface both

difficult to read and overwhelming to use. If we put only a subset of the possible criteria

for consideration, though, we are making choices about the criteria that the decision maker

should

use—clearly an inappropriate function for a designer of a DSS. Even if we list

all possible criteria but use multiple screens to display them, we are suggesting a relative

importance of the criteria by the order in which they are listed.

Hence, the goal is to summarize and guide while still allowing a great deal of flex-

ibility. One possibility is to categorize criteria and ask users first to specify the

How to use Perusall notes